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Mean values, traditionally used as a loca- 
tion parameter in the analysis of inter- 
comparisons, are known to lack stability 
against the effect of "outliers". It is 
therefore proposed to replace (or comple- 
ment) them by the use of medians, 
which have better statistical "robustness". 
An estimate for the corresponding uncer- 
tainty is derived and the procedure is illus- 
trated by a numerical example. The sim- 



plicity of the suggested robust approach 
should favor its practical use in a number 
of metrological applications. 
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1. Some Generalities 



The use of mean values as a location parameter has 
long been a deeply entrenched habit which scarcely 
requires justification. Also, by applying the principle of 
"least squares," one can even prove that this procedure, 
in some specific sense, leads to the "best" choice that 
can be made. 

For most users of elementary statistics it comes as a 
surprise, therefore, to learn that the very foundation of 
using mean values may come under question. Indeed, a 
rapidly developing branch of modern statistics, that 
which studies "robust" methods of estimation, has con- 
cluded (for quite some time already) that interpreting 
measurement results in terms of mean values is actually 
not a "safe" procedure because such values have poor 
stability against the effect of "outliers" (for a general 
review, see, for example, Ref. [1]). 

Outliers have been known for long, of course, but they 
were usually considered a nuisance in statistics — mainly 
because nobody really knew what to do with them. Yet, 
their incidental occurrence is a well-established fact. 



They pose a problem which seems to have no satisfac- 
tory solution. In principle, there are three ways to deal 
with outliers: 

• leave them stay as they are, 

• correct them, or 

• delete them. 

Unfortunately, all these procedures have unwanted 
features. Thus, if outliers are retained, they falsify both 
the mean value and its uncertainty, possibly to the point 
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at which the mean and its uncertainty become unaccept- 
able. Correction or deletion, in practice often performed 
by applying some more or less obvious weighting proce- 
dure or rejection rule, would require a clear justification. 
Hence, whatever procedure is followed, it is easily criti- 
cized. 

To this uncomfortable situation comes an unexpect- 
edly simple solution. It is offered by the expanding field 
of "robust statistics," and comes as a most welcome 
rescue. 

Let us concentrate on a single way — certainly the 
simplest and no doubt one of the more efficient ones — to 
achieve protection against the unwanted effects of out- 
liers. It is simply to replace the mean value by the 
corresponding median (or central value). For a continu- 
ous variate x, the median in is defined, using the (cumu- 
lative) distribution function F(x), by the condition 



F(m) = 7 2 . 



(1) 



This means that one half of the observations are below 
and the other half above the median. 

For a sample of n ordered variables X\, x 2 , ..., x„, the 
sample median, denoted as m = med {x, } , is given by 
(with integer k) 



Xk+\, 

l h(x k + x k+ i), 



k = — - — for n odd 



k = - for n even (2) 



As is well known, the median can also be obtained as 
the solution from the condition that 



The required estimate for the uncertainty of in is then 
taken as 



s(m) = C-MAD, 



(5) 



with a proportionality factor C which has to be evalu- 
ated. 

The constant C is determined by requiring that, in the 
limit of large samples, the estimate coincides with what 
we would obtain for a sample taken from a normal 
population. This is an arbitrary but reasonable normal- 
ization. 

The goal is achieved in two steps. First, we establish 
a relation between MAD and the parameter a (standard 
deviation) of a normal distribution, and then we use the 
known ratio of the variances for the median and the 
mean, both for a sample of size n taken from a normal 
population. 

For a normal distribution, the probability density 
function is 



<K*) = - 



l 



As for any symmetrical distribution, mean and me- 
dian coincide, thus rh = /jl. In addition, we can choose 
fi = without loss of generality; thus MAD = medflxl}. 
According to Eq. (1) we then have to evaluate the limits, 
— a and + a , for which 



4>(x)dx= 1/2, 



2j\xi — ih\ = minimum. 



(3) 



This equation then takes the place of the traditional 
principle of the least mean squares (see Appendix A). 



2. Uncertainty of the Median 

While the replacement of the mean value by the cor- 
responding median in is a simple and straightforward 
procedure, the estimation of the uncertainty s(m) to be 
associated with in requires some more thought. In the 
spirit of our robust approach we base this estimate also 
on a quantity which involves medians. An obvious 
choice is to use the "median of the absolute deviations", 
often abbreviated by MAD (a rather unfortunate choice), 
and defined by 



cf)(x)dx= 1/4. 



Tables give the numerical solution (for <t= 1) 



a = 0.6745. 



(6) 



We thus find that MAD can be linked, for a normal 
distribution, with the average standard deviation <r(x) of 
a single observation x, by 

MAD = a a(x). 

By increasing n , the precision of MAD is improved 
but its value remains essentially unchanged. Only for the 
special case of n= 1 do we always have MAD = 0. This 
can be taken into account by writing 



MAD = med {\x, - ml}, for i = 1, 2, 



(4) 



MAD- a/- -aa(x). 



(7) 



552 



Volume 105, Number 4, July-August 2000 

Journal of Research of the National Institute of Standards and Technology 



However, we must not forget that the uncertainty to 
be determined is that of a median, not of a mean value. 
From the theory of order statistics it is known that, in the 
case of a normal distribution, the (asymptotic) variance 
of the median, based on a sample of n values, is given 
by (see, for example, Refs. [2] or [3]) 



s\m) = y o- 2 (x). 



(8) 



Therefore, the uncertainty to be associated with the 
sample median in is 



s(m) = 



MAD 1.858 
' n - 1 a Vn - 1 



MAD. (9) 



In other words, the required proportionality factor in 
Eq. (5) can be taken as 



C 



1.9 



Vn - 1 



Note that Eq. (9) disagrees with a corresponding re- 
sult recently given in Ref. [4], where C is simply taken 
as 1/a. 



3. An Example 

To illustrate with a numerical example, we choose the 
six half-life measurements for 125 I performed in the 
framework of a recent international comparison of ac- 
tivity measurements [5]. The results, obtained in differ- 
ent laboratories, are (in units of days and ignoring the 
stated uncertainties), when arranged in order of increas- 
ing values, 

59.26 59.29 59.38 59.39 59.40 59.90. 

This leads to the sample median 

m = 72(59.38 + 59.39) d = 59.38 d. 

To determine the uncertainty of in, we list the abso- 
lute deviations of the values from their median, again in 
increasing order, i.e. 

0.00 0.01 0.02 0.09 0.12 0.52. 

This gives for their median, according to Eq. (4), 

MAD = 72(0.02 + 0.09) d « 0.06 d. 

Thus we have from Eq. (9) 



s (m) = ^ MAD = 0.05 d. 

V5 



The resulting estimate for the half life of 125 I is there- 
fore 



(59.38 ± 0.05) d, 



(10a) 



which compares favorably with the latest adjusted value 
of Ref. [6] 



T m = (59.408 ± 0.008) d. 



(10b) 



A traditional analysis (without weights) of the six 
values gives the mean value (59.44 ± 0.10) d, whereas, 
after deletion of the highest value as a possible outlier, 
one finds (59.34 ± 0.03) d. 

It will be noted that the suggested robust estimation 
method is extremely simple to apply and, in our exam- 
ple, leads directly to a reasonable result. Obviously, 
there exist more sophisticated approaches with a some- 
what higher efficiency (see, for example, Refs. [1] or 
[4]); their justification, however, is much less obvious 
and is not always free of subjective decisions. As a start 
in planned applications, the use of the simple method 
based on the median should be adequate. 



4. Remarks on Applications 

An important task of the BIPM is to organize and 
analyze international comparisons in the various fields 
of its activity. Traditionally, an essential part of such an 
exercise is the evaluation of a mean value (or reference 
value) with its respective uncertainty. Experience shows 
that the occurrence of discrepant results (outliers) is a 
rather frequent nuisance for the analyzer. While it may 
be necessary to neglect some data to protect the major- 
ity of participants from a misinterpretation, it is an un- 
pleasant task to inform a national laboratory that its 
result must be eliminated. Obviously, the Consultative 
Committees of the Comite International des Poids et 
Mesures (CIPM), which organizes such comparisons, 
would prefer to avoid such decisions which may cause 
problems to laboratories. 

As we have seen above, an analysis based on the 
median is largely insensitive to the existence of outliers 
(and their position). This is why we suggest that the new 
technique be applied, perhaps simply as a complement 
to the traditional analysis, in all international compari- 
sons organized by the Consultative Committees. In situ- 
ations without outliers, the additional result may serve as 
a welcome check. 
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Clearly, the analysis of an intercomparison largely 
depends on its purpose. While the determination of a 
consensus value is often the objective, in other cases the 
main interest is on discrepant data. Thus, for example in 
radiotherapy, all results within a given margin (for in- 
stance ±2 %) may be equally acceptable, whereas those 
outside pose a serious problem, as such irradiations are 
either useless or dangerous. For such results, the aim is 
to find a reliable location with respect to a stable refer- 
ence value, such as the median. 

It will be obvious that the use of a robust analysis for 
data of heterogeneous origin has a much wider field of 
application than intercomparisons. Similar problems oc- 
cur each time a compiler tries to determine a "best 
value" for application in physics, chemistry, or technol- 
ogy. In particular, the technique should also prove useful 
in the analysis of data on fundamental constants. 

If the data to be compared are not produced simulta- 
neously (or "blindly," as in an intercomparison), but are 
assembled over a period of time, additional problems 
occur since it is unrealistic to assume that they remain 
independent. Strongly discrepant results are normally 
not published. While the resulting distortion may have a 
moderate effect on the adopted mean value, such 
"psychological" correlations inevitably lead to an under- 
estimation of the uncertainty of input values for an ad- 
justment, possibly by a factor of two. 

An extension of the discussed robust approach to data 
with different statistical weights is possible, but not con- 
sidered an urgent task since the process of selecting 
such weights is usually subjective in nature. 

The above remarks are clearly of a personal nature 
and should not be taken as an official BIPM policy in 
these matters. 
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5. Appendix 



y-k == y-M 



■yo 



yt-i sy t , 



where k = (n — l)/2 for n odd, but without y and with 
k = nl2 for n even. 

Let us consider the various possibilities. 



For n = 2 



<r- t 
h 



y-i 



Vl 



If t is located between y- x and y t : Q = y-\ — y\ = <2o- 
For t outside this region we have 

Q = Q Q + 2\t- yi \ or Q = Q + 21? - y_,|. (A2) 

The minimum Q is reached for any t in the first 
configuration. One can choose t - (yi + y-i)/2. 

In the more general case of n even , an equal number 
of measurements is added to the left (y_ 2 , y-3, ...) and to 
the right (y 2 , y^, ...) of the interval considered above. 
Hence, the minimum 



Q = Q + I J (y J -y-j) 

still applies for t between y- t and y lt as for n = 2. 



(A3) 



<— t 



For n = 3 : 



y-i 



H 1 h 

y« yi 



In this case we obviously have (for t between y- x and 



y.) 



Q-y-i -yi + \t- y \ 

= y-i - yi = Qo, if t -y . 



(A4) 



This feature remains if additional results of measure- 
ments are added symmetrically (as above). The mini- 
mum of Q thus corresponds to the choice t = x for any 
odd value of n . The cases considered for n even or odd 
confirm the rule stated in Eq. (2). 



The purpose of this appendix is to show in a simple 
and explicit way that the sample median t = in is indeed 
the solution for the condition. 



Q = ^\xi-t\: 



■■ minimum 



(Al) 



for a sample of n results x, . 

It is practical to consider for this purpose the mea- 
surements Xi in their ordered form, say 
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